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ABSTBACT 

This oanual provides a suggested outline for a series 
of three workshops evolving fros the Pennsylvania Educational Quality 
Assessnent's "School Beport: A Status Profile** and "Hanual for 
Interpreting School Be ports** (both based on the Ten Goals of Quality 
Education) • In addition to the vorkshops, the following are 
discussed: (a) Act 299—the School District Beorganization Act; (b) 
isplesentation of Intervention techniques to incr'^ase goal scores; 
<c) the Ten Goals of Quality Educations (d) instru ^nts, scoring, and 
variables used to seasure goals; and (e) isplioations for the 
curricultti. Also included is a set of 23 plates, profiling the 10 
goals, raw scores, norsative referencing, criterion referencing, and 
condition variables, which can be used as transparency sasters. 
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TEACHER WORKSHOP MANUAL 
''EDUCATIONAL QUALITY ASSESSMENT - PROFILE REPORT" 

Preface 

The Pennsylvania Educational Quality Assessment's School Report: 
f Sl^fttua Profile is a comprehensive report on both mean student scores 
on Pennsylvania's Tan Qoals of Quality Education and related condition 
variables measured in the school district* It is a document that 
cannot be easily assimilated. The mean student achievement scores are 
reported in four different ways to enable school district personnel to 
properly evaluate their educational system* It is suggested that one 
professionali say the district's curriculum coordinator or assistant 
superintendent! become familiar with both the Profile Report and the 
Manual for Interpreting School Reports in preparation for conducting 
a sexdes of three workshops described in this brochure. These 90-miuute 
workshops are designed to inform the teaching staff, in a clear and 
concise manner, of the contents of the report* 

Included in the manual is a suggested outline for the workshops « ' 
descriptive material to supplement the EQA Manual for Interpreting School 
Reports, and a set of 23 plates that can be used as transparency masters* 
Actual school district data may be written on the appropriate transparencies 
with transparency markers* In addition to the plates « page 6 may 
be removed and used as transparenoy masters for the 10 goals and 

pages 4 and 5 for Act 299* 

There are no educational panaceas that will guarantee an Improvement 
in any of the EQA goal areas* A necessary component of currlculuir. 



revision based upon the report is faculty understanding and aoceptanoe 
of the Tan Goals of Quality Education , as wall as the data contained 
in the report. The awareness and involvement of the faculty are paranount 
for any intervention technique to en;*oy a significant probability of 
success • 



OUTLINE OF WORKSHOPS 



First Meeting 

1. Discuss Act 299 and its ramifloatlons. 

2. E3q}lain flow chart from EQA testing to the implementation 
of intervention techniques (Plate XIII). 

3* Discuss the Ten Goals of Quality Education and their 
measurement (hand out copies of the goals to each 
vforkshop participant)* 

A. Discuss the goal subsoales using Information in the 
EQA Manua l for Tnt-ftvp^ntMn^ School Reports . 

5* Discuss the four methods of reporting the BQA scores 
including sample condition variables using information 
in the EQA interpreter's manual. 

6. I^esent the criterion-referenced scores for the affective 
goals and the distributions for Goals 111, VI (grade 5) 
and VIII-K (perhaps only five goals at the first 
meeting and the remaining five goals at the second 
meeting). En^hasize that the saorple items^ a«e«f Goal 1, 
subscale Self-Confidence: **I*m pretty sure of qyself 
is only a sanrple of the type of items used to measure 
a subscale. In general » several items are used for 
each subscale. 



Review the four methods of reporting the EQA scores* 
Review the criterion-referenced scores for the district. 



Second Meeting 

1. 
2. 



3* Present the raw school meansi state peroentllesi predicted 
scoresi and the con^arlson of district scores with 
predicted scores. Also include the high, low and mean 
state scores from the appropriate EQA interpreter's • 
manual* 

4* Show correlation matrix of goals with goals. Some 

discussion of the theory behind correlation coefficients 
may be necessary (Appendix A in Manual Two) • 

Third Meeting 

1. Review predicted range and local school goal scores* 
Some discussion related to prediction may be appropriate 
(see pp. 1->4| Appendix A in Manual Two). 

2. Describe condition variables and present local school 
data and statewide data (maximmni minimum and mean) on 
these variables. This might include the identification 
of the predictor variables for each goal, as well as 
information £rom the matrix of correlation coefficients 
between the condition variables and the goal scores* 
These data are contained in the appropriate EQA Manual 
for Interpreting School Reports . 

3. Ejiplore the implications of the report to the ourrioulum. 

ACT 299 

Act 299 (1963) > the School District Reorganization Aot, was designed 
to encourage small school districts to combine into larger districts for 
the pitrpose of improving education. Some small districts i however, 

n 
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maintained that larger jointures would only reduce the quality of 

educati.on available for their students. In an attempt to define quality 

education and to provide a method of measuring such education, the act 

required the State Board of Educations 

"•••to develop or cause to be developed an 
evaluation procedure designed to measure 
effectively the adequacy and efficiency 
of the educational programs offered by the 
public schools of the Commonwealth.. tThe 
evaluation procedure shall be so constructed 
and developed as to provide each school 
district \dth relevant comparative data to 
enable directors and administrators to more 
readily appraise the educational performance 
and tc effectuate without delay the strengthening 
of the district's educational program*.." 



PROM EQA TESTING TO IMPLEMENTATION OF INTERVENTION TECHNIQUES 



Act 299 was a mandate for the Department of Education to develop a 
statewide assessment program that would provide school districts with 
useful comparative data for the purpose of improving a district's 
educational program. As a result, Pennsylvania's Ten Goals of Quality 
Education were written, measuring instruments were developed, and a 
procedure for reporting the results to the school district administration 
was established. The latter included a report called ^uoatlonal Quality 
Assessment . School Report i A Status Profile . \Mch is presented to the 
administration by a team from the Pennsylvania Department of Education. 

A model that may be useful in progressing from the EQA testing, 
through a needs assessment and culminating in the implementation of 
intervention techniques designed to increase goal scores, is represented 
on Plate XXIII. This manual and the set of workshops described herein are 
an integral part of that model. 
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THE TEN GOALS OF QUALITY EDUCATION 



The initial task facing the department as a result of Aot 299 was to 

define quality education. This was accomplished by the establishment of 

the following Tan Goals of Quality Education i 

I« Quality education should help every child acquire the 
greatest possible understanding of himself or herself 
and an appreciation of his or her worthiness as a 
member of society. 

II. Quality education should help every child acquire an 
understanding and appreciation of persons belonging 
to other social » cultural and ethnic groups. 

III. Quality education should help every child acquire » to 

the fullest extent possible » mastery of the basic skills 
in the use of words and numbers. 

IV. Quality education should help every child acquire a 
positive attitude toward the learning process. 

V. Quality education should help every child acquire the 
habits and attitudes associated with responsible 
citizenship. 

VI. Quality education should help every child acquire 
good health habits and an understanding of the 
conditions necessary for the maintaining of physical 
and emotional uell-being. 

VII. Quality education should give every child opportunity 
and encouragement to be creative in one or more fields 
of endeavor. 

VIII. Quality education should help every child understand the 

opportunities open to him or her to prepare for a productive 
life and help each child to take full advantage of these 
opportunities. 

IX. Quality education should help every child to understand 
and appreciate as much as possible of human achievement 
in the natural sciences » the social sciences » the 
humanities and the arts. 

X. Quality education should help every child to prepare for 
a world of rapid change and unforeseeable demands in vMoh 
continuing education throughout adult life should be a 
normal escpectation. 



Note that eight of these goals are in the affective domaini i.e., 
related to attitudes, while only two are cognitive, i.e., related to 
knowledge . 

A more succinct picture of each goal may be presented by calling the 
goal subscales "components" and giving a brief description of each. The 
EQA Manual for Interpreting School Reports and the technical manual. 
Getting Inside the EQA Inventory , contain subscale information} however, 
the descriptions on Plates I through XIII may suffice. 

THE INSTRUMENTS 

With quality education thus defined, 10 instruments were developed 
to measure the goals. Most of the instruments were further divided into 
subinstruments or subscales In an attempt to better assess the goals* 
For example, the 1973, d4-item instrument used to measure citizenship 
(Goal V) had three subscales t 

1. Welfare and Dignity of Others (15 items) 

2. Respect for Law and Authority (21 items) 

3. Responsibility and Integrity (18 items) 

Statistical data related to the instruments and their rationale are 
discussed in the EQA publication, r,»t.t^tr>f^ Tr\a±d& the BQA Inventory. 

SCORING 

The problem of scoring and reporting measurements made by any scale 
or test is formidable. EQA reports the data in four distinctly different 
ways, each of which reveals a unique bit of information. All four reporting 
methods must be considered vdien analyzing the EQA data and should be 



discussed at eaoh vforkshop session. These methods are described below 
using a fictitious counselor and a hypothetical, counselor-made , self- 
esteem scale* 

Rav Score ("School Score") 

A school guidance counselor may decide to measure the 
self-esteem of the students in the district by vndting 10 
items that seem to relate to a student's self-esteem. The 
10-item instrument is administered and scored and a mean 
school score of, say 7*2 items, is calculated. This is 
merely a raw score mean (or average) and by itself is rather 
meaningless. For example t Is this a good self-esteem score or 
not? Kow does this compare with students in an adjacent 
district? How does this compare with students from 
schools similar to this school? Hcv doos this cosrpare with 
student" s tatewide? How truny of our students have a very 
positive self-esteem? 

These questions cannot be answered from the raw score 
school average. They can, for the most part, be answered 
from the other methods used for reporting in the EQA ^tatus 
Profile . 

Note also that the instruments have different numbers 
of items and varying weighting procedures for each goal. 
For example, in grade 8, the Qoal I soaJe contained AO items 
and had a maximum possible score of 120, iMle Goal II scale 
contained 35 items and had a maximum possible score of 105. 
Raw scores cannot be compared across goals. 
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state Rank or Peroentlle ("Sohool $lle") 

The sohool guidance counselor (School A) may decide 
to K4k colleagues in 11 schools throughout the state to 
administer the instrument and report the results* If 
the school mean scores were then ranked, the follovdng 
picture might emerge t 



SCHOOL :GORE RANK 

E 8.9 1 

F 8.7 2 

D 8.6 3 

L 8.4 4 

G 8.3 5 

I 7.9 6 

K 7.6 7 

B 7.5 8 

A 7.2 9 — 33rd percentile 

J 7.1 10 

H 6.9 n 

C 6.4 12 ' 



The school in questioni Sohool A, ranks 4th from the 
bottom (or 9th from the top) of the sample of 12 schools. 
This is the 33rd percentile. In other words , Sohool A 
scored at or above 33 per cent of the 12 schools tested 
on the lO-item self-esteem instrument > and below 66 per cent 
of those schools tested* 

Percentile rankings must be viewed in proper perspective 
to avoid misinterpretations. The EQA procedure uses turn-around 



norms, i.e., each aohool is part of the normative sample, 
a process which, by definition, means that half of the 
schools in the sample must necessarily fall below the 50th 
percentile. This point may be accentuated by noting that 
half of the practicing medical doctors in the state were 
graduated in the bo«.><om half of their class. 

Rank Comoarad to Similar Schools ("Predicted Score Range") 

The guidance counselor may decide to ask colleagues 
in eight schools ••similar" to his/hers to administer 
the 10-item scale. ••Similar^^ might be oased upon 
school size, education of teachers and occupation of 
parents. If the mean scores from these nine schools 
were ranked, the following picture might emerges 



SCHOOL SCORE RANK 

I 7.9 1 

B 7.5 2 

K 7.3 3 

7,2 4 — 641bh percentile 

J 7.1 5 

p 7.0 6 

N 7.0 7 

H 6.9 d 

0 6.4 9 



When compared with '• similar •• schools. School A 
ranks in the top half. Another way to describe the 
result is that ••the range of most of the scores from 
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slfflllar schools Is 6.9 to 7.5 and the district in question 
scored 7.2 which is within this range." 

This method of matching schools and identifying an 
• expected range is not the procedure used by EQA} however^ 

the end product may be viewed in this manner. Actuallyi 
EQA uses multiple regression prediction equations generated 
annually from the current year's sample. 

Criterion-Re f erencing 

The guidance counselor may decide that if a student* 
answers one more than half of the items in a favorable 
vmy with respect to a positive self-esteem) then the 
student haS) in fact} a positive self-esteem. T.'3 student 
response sheets are then rescored and those students having 
six or more of the 10 items answered favorably are considered 
to have **passed** the scale. If this is done, the following 
picture may emerges 

School A Seventy-two per cent of the students answered 
more than half of the 10 items in a favorable 
way - i.e., **passed the self-esteem scale** (leaving 
28 per cent of the students with a poor self-esteem). 

Total of Other 

Schools Fifty-nine per cent of the students **passed 

the self-esteem scale." 
In addition, the data for the per cent of students 
**passing** using a 35 per cent criterion, and the per cent of 
students "pasfling" the 70 per cent criterion are also listed 
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in the profile. Both individual schools and statevide data 

are presented. The 51 per oent criterion is not only listed but 
graphed for each of the affective goals and their subscales. 

For Goals IIIi VI (grade 5) and VIII-K» approx5jnate qulntile 
score distributions rather than criterion-referencing are 
printed. This method of displaying data permits the 
identification of the per cent of students statevide getting 
11 or less items correct; 12 to 15 items correct; 16 to 18 
items correct; 19 to 22 items correct; and 22 or more items 
correct for Goal III, grade 11 (1974)* Goal III-M, VI (grade 5) 
and VIII-K are similarly reported* 

CONDITION ViAIABLES AND PREDICTOR VARIABLES 

About three dozen variables that may have predictive value vhen used 
to "match" schools were identified by EQA* These variables , such as 
percentage attendance, father's occupation and education of the teachers, 
are used in a multiple regression analysis to predict how a school might 
perform on a given goal* The predictions are based upon current data 
from many schools. The process is similar to predicting a show siee 
from knowledge of a person's height and weight (see Appendix A, Manual Two)* 
A certain subset of the condition variables, usually the ones having the 
greatest correlation with the goal in question, are used in a prediction 
equation* VAien used in this manner, the condition variables are called 
predictor variables* Each of these variables is described in the j^Qi^ 
for Interpreting School Reports along idth certain data oolleotlen forme 
and statewide averages on each variable* 

12 



Furthermore, the matrix of correlation coefficients between these 
variables may be found in the Manual Sss. Interpreting School Report>s« 
Be careful about the condition variables: 

1. It may well be that there are no other srhools exactly 
like yours when compared using the predictor variables. 
The best that can be said is that schools similar to 
yours on these variables would be expected to score within 
the identified Predicted Score Range. There is no way 

to delineate those \diich are similar on these variables. 

2. No cause and effect relationship is inferred by the 
correlation matrix. If percentage attendance is used 
in part to predict a goal score, one cannot say a low 
percentage attendance has caused a low goal score. 
Certain hypotheses may be developed, however. 

3. All data are mean school data and the correlation coefficients 
are based on these school means. A significant positive 
correlation coefficient implies that schools high on one 
variable tend to be high on the other. It does not 

insure the same relationship for individual students. 
Transparencies reproduced from Plates XVIII and XXII in this manual 
may be used to present the local school's condition variable data. 

IMPLICATIONS FOR THE CURRICULUM 

If, as a result of the EQA Assessment Program, it is determined that 
several goal areas need attention, a plan of action could be discussed 
during the last workshop. With a plan of action determined, the 

13 



EQA Manual for the Identification of Crltloal Qoals should be consulted. 
This contains a model for the prioritization of goals based upon the 
school district philosophy and the data contained in the School Report 
and EQA documents written specifically to assist districts with each 
goal area* These documents y some of vhich will be available in V^tUt 
contain pertinent information relating to educational parameters and 
instructional strategies appropriate for each goal* 
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